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It is observed that for testing between simple hypotheses where the cost 
errors can be quantified, it is better to let the optimization choose the test 
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I. HYPOTHESIS TESTING 


Let (X, T, p) be a cr-finite measure space and let V be the family of probability measures 
P on (X, T) which are absolutely continuous with respect to p so that, for A £ T, 



Here p = dF/dp is the density (Radon-Nikodym derivative) of P with respect to p. We are 
mostly interested in two cases: The first is when X is a Euclidean space M iV equipped with 


the Borel cr-held and p is Lebesgue measure. The second is when X = 7L N or X = and 


p is counting measure on all subsets of X. This allows us treat probability densities and 
discrete probability distributions simultaneously. 

Let P 0 ,Pi G V and let p 0 and pi be the corresponding densities with respect to p. Let 
(Ad,... ,Xn) be the available sample taking values in X. We seek a test tp : X —> {0,1} 
such that, if (aq,..., aqv) are the observed values, </?(aq,..., xn) = 0 if we accept H 0 = {Po} 
and (p(x\,... ,xn) = 1 if we accept H\ = {Pi}. Let C be the critical region, namely the 
subset of observations x = (aq,..., xn ) such that </?(aq,..., x^) = 1, namely where we reject 
the null hypothesis, cf. e.g. [1, Chapter 8]. 

II. A CLASS OF INFERENCE PROBLEMS 

Consider a simple hypothesis testing problem where we can quantify the cost of each 
error. Namely, if we reject H 0 when it is true we incur the cost Cq > 0 and if we reject 
H\ when it is true we incur the cost C\ > 0. This is the case in many applications such as 
when, on the basis of a sample, we need to decide whether to halt the production of an item 
which should meet certain required standards. Both producing a whole stock not meeting 
the requirements or halting the production process when the requirements are met causes 
certain quantifiable costs. A type / error occurs with probability a = P 0 (C) while a type II 
error occurs with probability /? = Pi(C c ). It is then natural to try to minimise the cost 


J{C) = c 0 F q {C) + c 1 F 1 {C c ). 


This is a simple unconstrained optimisation problem which can be formalized as follows. 
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Problem 1 Find a measurable set C C X such that the following cost function 


J(C) = c 0 P 0 (C) + ciPi(C c ) = J [c 0 p 0 (x ) - cipi(x)\ dp + ci 

is minimised or, equivalently abusing notation, minimize 


J(l c ) = f t c [coPo(x) 
Jx 

where 1 c is the indicator function of the setC. 


cipi(x)\ dp 


Let us introduce the set 


Q = {feL°°(X,F,p)\f:X^[0,l}}, 


and consider the following “relaxed” version of Problem [0 


Problem 2 

Minimiz e feQ J(f), 


where 

j(f)= / f( x ) [c 0 Po(x) - CiPi(x)] dp. 

Jx 

Observe that the cost function is linear in / and Q is convex. Thus, this is a convex 
optimization problem. We recall a few basic facts from convex optimization. Let K be a 
convex subset of the vector space V, let F : K —y M be convex and let Xo G K. Then, the 
one-sided directional derivative or hemidifferential of F at Xq in direction x — Xq 


Ff(x 0 -,x 


x 0 ) := lim F{x ° + ~ Xo)) - 

eX^O 6 


exists for every x G K (this is a consequence of the monotonicity of the difference 
We record next the characterisation of optimality for convex problems, see e.g. 


quotients). 

[2, p.66]. 


Theorem 3 Let K be a convex subset of the vector space V and let F : K —> R be convex. 
Then, Xq G K is a minimum point for F over K if and only if it holds 

F' + (x 0 ; x — xf) > 0, Wx G K. (1) 

We can then apply this result to Problem [2] 
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Proposition 4 The minimum in Problem Q] is attained for 


C* = {x e X\c 0 p 0 (x) < cipi(x)}. 


( 2 ) 


Proof. We apply Theorem [3] to Problem [2] and get that a necessary and sufficient condition 
for f* e Q to be a minimum point of J(f) over Q is 


J'(f*',f-f*)=[ [f(x)~ f*(x)][c 0 p 0 (x)-c 1 pi(x)]dfx>0, ifeQ. (3) 

J X 


Observe now that f* = 1 c* satisfies d^J). Indeed 



since both integrals in the last line are nonnegative. Indeed, f(x) — 1 < 0 and, on C*, 
CoPo(x) — CiPi(x) < 0 imply that the integrand in the first integral is nonnegative. The 
integrand of the second integral is the product of two nonnegative functions and is therefore 
also nonnegative. Finally, since f* = 1 c* is an indicator function, it also solves Problem [Q 
□ 

Remark 5 We can rewrite the optimal critical region in the familiar form 



(4) 


Thus, the ratio of the two costs cq/c\ plays the role of the multiplier associated to the size 


constraint in the usual Neyman-Pears on approach. The size of the test and its power, are 
simply 



III. EXAMPLE 

We illustrate this approach in the simple case of testing the mean of a normal dis¬ 
tribution with known variance. Let /i be Lebesgue measure on M, p 0 = J\f( 0,36) and 
Pi = A/”(l-2, 36). Suppose (xi,... ,xjy) are the observed values from a random sample and 
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let xn = (1 /N) x i be the sample mean. Let us fix a = 0.05 and let N = 100. Then the 

optimal Neyman-Pearson test has critical region Cnp = {xioo > 0.987}. The corresponding 
error of the second type is (3 = 0.36. Since only the ratio (co/ci) matters in the minimisation 
of Problem [TJ we take from here on c\ = 1. Thus applying the Neyman-Pearson approach 
with tests of size 0.05, we incur the cost 

J[C NP ) = c 0 (0.05) + 0.36. 

Next, we compare J{C N p) with J(C*) = c 0 a* + /?*, with C* given by © and a* and /3* given 
by (J5]), for different values of Co and c\ = 1. We get the results of Table |U 

TABLE I: Comparison of costs 

j(c NP )Jin 

0.05 + 0.36 = 0.41 0.1587 + 0.1587 = 0.3174 

2.718 x 0.05 + 0.36 = 0.495914 2.718 x 0.06681 + 0.30854 = 0.490129 

7.387 x 0.05 + 0.36 = 0.7293762 7.387 x 0.02275 + 0.5 = 0.668066171 

20.07929 x 0.05 + 0.36 = 1.3639645 20.07929 x 0.00621 + 0.69 = 0.81469239 


c 0 = 1 
c 0 = e 
c 0 = e * 1 2 
c 0 = e 3 


We see that in all cases, as expected since C* gives the minimum cost, fixing a a priori 
without considering the costs of type I and If errors, leads to a higher cost. The costs are 
closer when a* is close to 0.05. Indeed, if a* happens to be 0.05, given the form 05) of C*, 
we have C* = Cnp- 

In conclusion, when the cost of the two errors is known, it appears wiser to let the 
optimization determine the size of the test through (l5]h 
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